Objectives

1. Describe the process used for General Education assessment at Ferris State University
2. Provide access to all of our assessment resources
3. Demonstrate several mechanisms for involving faculty in the assessment process

Presentation Handouts

• PDF versions are available online
  
  

• QR codes and website information

• Training handout and standard measures

About Ferris State

General Education Assessment

• Goal: Track and improve program effectiveness
  

  • Our program primarily consists of courses
  • Embedded course assignments are the most meaningful and valid measures of student learning.
  • General Education applies to all students
    

• Therefore, we are measuring student performance in General Education using embedded assessments for all students in all courses.

Project Scope

• In a typical semester, General Education at Ferris:
  

  • Offers over 400 different courses
  • Instructs over 5,000 individual students
  • Involves over 500 different faculty
  • Generates over 50,000 student credit hours
  • Is responsible for $20,000,000 in revenue

Project Resources

• Program coordinator (me, 50% release)
• General Education Committee (advisory, no release)
• Line item budget ($0)
• Dedicated staff (none)
• Other staff (Registrar and IR)

The Assessment Cycle

The Futile Cycle

A Better Model

“Assessment is not a spreadsheet – it is a conversation”
Irmeli Halinen

Reproducible assessment

Our goal is to create reproducible processes to increase transparency and encourage faculty engagement.

• ALL assessment resources are public-facing
  

  • Data reports (PDF and HTML)
  • Data analysis code (R)
  • Deidentified data files (csv)
    

• All released with MIT or CC-BY licenses
  
• All results are computationally reproducible and extensible

Striking a balance

Sustainable assessment

• The three keys to our program assessment strategy:
  
  

1. Simplification — reduce complexity
   
2. Standardization — reduce variance
3. Automation — reduce workload

Simplify: Core Competencies

• Eight competencies:
  

• Collaboration, Communication, Culture, Diversity, Natural Sciences, Problem Solving, Quantitative Literacy, Self and Society
  
• Each overseen by a faculty subcommittee (6-10 members)
  
  

1. Operational definitions
   
2. Goals (Hallmarks of a Bulldog)
   
3. Relevance statements

Simplify: Ferris Learning Outcomes

• Each competency has four learning outcomes
• Based upon what we want our graduating seniors to look like
• Derived from LEAP
• One outcome measured each semester
• Rolling two-year program cycle

Standardize: Rubrics

• Derived from VALUE rubrics
  
• General Education now has a uniform:
  
  

1. Scoring system (zero to four point scale)
   
2. Analysis interpretation (four threshold values)
   
3. Criterion leveling (expectations by course level)

Standardize: Measures

• Created in order to get more uniform data collection and summarization

• 14 different type of measures are defined
  
• Instructors can choose to use any one they wish
  
  

1. Exams
2. Products
3. Performances

Standardize: Registering Courses

• I need to know who is doing what each semester

• Who is collecting data?
• What competency is being measured?
• What outcome is being measured?
• What measure is being used?
• All tracked using a Google form

Standarize: Data Collection

• This is the biggest step forward for us

• Excel workbooks to collect and summarize student results
• SCORE, PREPOST, NORM, RUBRIC
• Common metadata supplied by instructors
• Enter student names, IDs, and appropriate evaluations
• A standaridized summary is automatically generated

Standardize: TracDat Assignments

• Seeking to minimize the need for faculty to log into TracDat

• Received as an email
• Copy and paste summary
• Select a few responses from drop-down menus
• Add a short reflection
• Attach the workbook and submit

Automate: Analysis of Student Data

• All workbooks are stored in the TracDat document repository

• Each semester’s data is in a separate folder
• Download all the workbooks
• An R script is used to process the workbooks
• All data is aggregated into one csv file
• Identifiable information is removed

Automate: Reporting

• Using R and bookdown (Rmarkdown)
  
• Reuseable, version-controlled code
  
• PDF reports for printing
  
• HTML reports for distribution and collaboration

Assessment in Action

• Five means of creating a dialog:
  
  

1. Print reports for dissemination
   
2. Online forums for extended discussions
   
3. OSF projects as the ultimate home base
   
4. GitHub repositories for reproducibility
   
5. Executive dashboard for overviews

Summary

• We are creating a collaborative assessment environment
  
• This is more about process than personality
  
• Focus is on making a resource rather than just reports

Questions?

• All of these resources are linked to from your handouts
  
• Licensed using either CC-BY 4.0 or MIT
  
• Feel free to contact me if you have questions or comments